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Widespread acceptance of technological 
literacy as a desirable outcome of education 
has led to the development and implementa¬ 
tion of a variety of curriculum innovations in 
the field of technology education. Learning 
activities may mask some degree of concep¬ 
tual fuzziness about the construct of techno¬ 
logical literacy (TL). However, as they use 
assessment strategies in their work, curricu¬ 
lum developers strive for utmost clarity and 
precision about the psychological states im¬ 
plied by the construct and about its overt 
behavioral manifestations. The questions to 
be answered in the process of developing 
strategies to evaluate TL are "What constitutes 
TL?" and "What knowledge, attitudes, and 
behaviors should a technologically literate 
person display?" These questions must be 
answered clearly and precisely if valid and 
appropriate assessment strategies are to be 
designed. 

This article draws on some recent commen¬ 
taries on these questions to suggest some ways 
these issues might influence the design of 
assessment strategies for TL. 

TECHNOLOGICAL LITERACY 

Dyrenfurth (1991) suggested that TL is "a 
concept used to characterize the extent to 
which an individual understands, and is ca¬ 
pable of using technology" (p. 139). Although 
there are almost as many definitions of this 
construct as there are authors defining it, the 
definitions all embody the notions of knowl¬ 
edge, and understanding, of technology, and 
capability in its use. However, when attempts 
are made to specify the knowledge and to 
identify the competencies implied in capabil¬ 
ity, it becomes apparent that these definitions 
are quite diverse. 

Yff and Butler (1983) suggested the purpose 
of TL is to enable citizens "to weigh alterna¬ 
tives and make informed decisions. It should 
enable [people to] manage their lives and 
cope with change to their best advantage" (p. 
14). In a discussion of the technology compo¬ 
nent of the British national curriculum, Farrell 
(1992) described its outcome as the education 
of young people "to be capable in a society 
where they are constantly interacting with the 
made world" (p. 40). This latter statement 
brings into focus the social, cultural, and his¬ 
torical dimensions of TL since the "made 
world" could be expected to vary across cul¬ 
tures and, over time, within cultures. 

This context-specificity makes it difficult to 


spell out the manifestations of TL in anything 
but general terms. Nevertheless, if appropriate 
assessment modes and strategies are to be 
devised, it is necessary that in each context the 
desired manifestations of TL be specifically 
determined. It is useful to categorize these as 
knowledge, skill, and affective manifestations. 
This broadly corresponds to the three dimen¬ 
sions used by Dyrenfurth (1991) who sug¬ 
gested that TL includes a civic dimension— 
the ability to understand the issues raised by 
technology; a practical dimension—the abil¬ 
ity to use technology; and a cultural dimen¬ 
sion—the appreciation of the significance of 
technology. Lewis and Gagel (1992) argued 
that the conceptions of TL vary with the philo¬ 
sophical orientations of the constituencies at¬ 
tempting to define it. It is in the relative impor¬ 
tance of the affective, cognitive, and skill 
components of TL that this diversity is most 
apparent. It should be recognized that these 
domains are not watertight and that any prac¬ 
tical manifestation of capability is in fact an 
amalgam of all three. 

Knowledge Component 

Lewis and Gagel (1992) argued that "at its 
core literacy implies knowledge" and further, 
that "levels of literacy seem to correlate with 
levels of knowledge" (p. 21). The concepts 
and understandings related to technology must 
be the knowledge content of curricula in TL. 

The broad knowledge areas identified in 
the literature are summarized as follows: 

1. A knowledge of problems that might have 

technological solutions. Yff and Butler 
(1983) suggested that a curriculum for TL 
should include a study of "the major so¬ 
cial, economic, and geophysical prob¬ 
lems" (p. 13). They include among these 
such problems as hunger, transportation, 
and waste disposal. 

2. A knowledge of important technologies 

such as computer applications, systems 
dynamics, industrial processes (Yff & But¬ 
ler, 1983), biotechnology, materials, and 
energy technologies (American Associa¬ 
tion for the Advancement of Science, 
1989). 

3. Understanding of the social and cultural 

impact of technology such as the effect of 
technology on societies, its value¬ 
laden ness, and its irreversibility (Heinsohn, 
1977; Mears, 1986). 

4. The range of concepts that are prerequisites 

for an understanding of technology drawn 
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from such other disciplines as science, 
mathematics, history, and language (Lewis 
& Gagel, 1992). 

5. An understanding of theform or structureof 
technological knowledge. This implies un¬ 
derstanding of knowledge of what works 
and therefore has a practical dimension. It 
also implies an appreciation of how tech¬ 
nological knowledge is related to other 
forms of knowledge, particularly science. 

Capability, which lies at the heart of TL, is 
essentially the ability to think and do effec¬ 
tively in the context of the real world. This 
implies a range of both cognitive and psycho¬ 
motor skills that Layton (1987) has character¬ 
ized as functional competencies. In addition 
to those cognitive skills that relate to ways of 
processing information, the technologically 
literate persons should display the ability to 
think critically about technology itself. 

Skills Component 

One concept of TL emphasizes the ability 
to evaluate technology as one of the core 
characteristics of the technologically literate 
person. Donelly (1992) described this view as 
a "small but important radical strand of thought 
about technology education" (p. 133). Lewis 
and Gagel (1992) suggested that the techno¬ 
logically literate person should "be able to 
fashion informed opinion regardingthe social, 
political, environmental, or economic conse¬ 
quences" (p. 131) of technological activity. In 
the same vein, Yff and Butler (1983) postulated 
that the most important aspect of TL is that "it 
should enable citizens to recognize when 
others, to whom they have entrusted the man¬ 
agement of their social institutions, are not 
acting in their interests" (p. 14). 

Engaging in technological activity is an 
important aspect of capability, and one that 
involves a complex interaction of cognitive 
and manipulativeskills. Schwaller(1989) iden¬ 
tified some of the cognitive ones as analytical 
thinking, creativity, problem solving, research, 
and analysis. The manipulative skills are those 
involved in the design process and in the 
makingof technological products. Design skills 
are central to technological activity. These 
skills must be broadly conceptualized to in¬ 
clude the abilities to recognize those prob¬ 
lems that might yield to technological solu¬ 
tions, generate ideas, and formulate strategies 
for implementing ideas. 

The Affective Component 

Layton (1991) argued for the "crucial con¬ 
ative" componentof technological capability. 
This refers to a willingness that must precede 
action in a technological or any other context. 


Kozolanka and Olson (1994) extended this 
component to the realm of virtue when they 
suggested there also needs to be the capacity 
to act for the right reasons. The profound and 
pervasive impact of technology on society 
makes this a critical issue. These relate to the 
question of social responsibility. The techno¬ 
logically literate person could be expected to 
exhibitnota mere awareness of, buta concern 
for, the "moral and ethical implications of 
technological choice" (Lewis & Gagel, 1992, 
p. 130). 

While education in technology should not 
be constrained by narrow vocational con¬ 
cerns, TL cannot be divorced from preparation 
for employment. There are at least two reasons 
for this. The first is that the concept of literacy 
is at its heart concerned with the competen¬ 
cies needed for functional adult life. The sec¬ 
ond is that the world of work is the major arena 
of technological activity. TL, then, implies the 
possession also of such affective workplace 
ski I Is as flexibility and team spiritedness, among 
others. 

It seems helpful to consider the discussion 
of the preceding three components in the 
context of the following summary of the con¬ 
struct of TL, which indicates that it: 

• is multidimensional; 

• implies a range of functional capabilities 

that include the designing and making of 
technological solutions to problems, the 
monitoring of the societal impact of tech¬ 
nologies, the evaluation of technology 
from a variety of value criteria, and the 
ability to use those technologies that are 
appropriate to one's own context; 

• impliesa knowledgeofsome basic concepts 

related to technology and an understand¬ 
ing of how certain technologies work; 

• is dependent on a range of cognitive skills 

such as analytical skills, creativity, brain¬ 
storming, problem solving, and data col¬ 
lection; and 

• has a major affective component including 

such attitudes as independence and inter¬ 
dependence, caring, environmental con¬ 
cern, social responsibility, and positive 
work habits. 

Thus we have a set of functional competen¬ 
cies that characterize TL. They are the compe¬ 
tencies that a technologically literate person 
could be expected to display in appropriate 
conditions. These would be used to guide the 
choice of approaches and the design of tasks 
for the evaluation of TL. 

The assessment of learning in technology 
should therefore be guided by a consideration 
of these functional competencies as well as by 
the following three important characteristics 
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of technology itself: (a) it is practical and 
grounded in the real world, (b) it is as much 
process as it is product, and (c) it is very much 
a way of thinking and acting in relation to the 
material world. 

SOME ASSESSMENT ISSUES 

Increasingly, the term assessment is being 
used to signify the change from an almost 
singular reliance on tests that gave quantifi¬ 
able results to methods of evaluation that 
recognize the complexity of human function¬ 
ing and that more closely reflect the real-world 
context of human performance. Indeed terms 
such as authentic (Wiggins, 1989), illuminative 
(Hodson & Reid, 1988), and expressive (Eisner, 
1993) are being used to describe assessment 
procedures that elicit a display of student learn¬ 
ing in its uniqueness and complexity. 

The use of student portfolios, group as 
opposed to individual tasks, performance as 
well as product evaluation, ongoing as op¬ 
posed to single-event assessment, and open- 
ended rather than closed tasks are strategies 
that make learning visible as it progresses and 
unfolds in its uniqueness for each learner. 
Such approaches serve well the purpose of 
engaging pupils in their own learning while 
providing diagnostic information to teachers 
and of affirming the individuality of each 
learner. It seems evident thatthese approaches 
to assessment are the ones that wi 11 provide the 
best evidence of technological literacy. 

Assessment strategies that value the idio¬ 
syncratic nature of student learning raise the 
issue of standards. Use of a common yardstick 
by which to measure individual outcomes has 
been one of the hallmarks of traditional forms 
of evaluation. This is in essence a validity 
issue. In a discussion of the nature of perfor¬ 
mance assessment, Messick (1994) suggested 
that "authenticity" and "directness," which 
are qualities claimed to be characteristic of 
performance assessment, are related to the 
issueofvalidity. FrederiksonandCollins(1989) 
and Linn, Baker, and Dunbar (1991) proposed 
that specialized validity criteria need to be 
invoked for performance assessment. The is¬ 
sueofvalidity as it relates to the assessment of 
technological literacy will be addressed again 
later in the article. 

Factors in Designing Assessment 

Valid assessment should be construct- 
driven. It is the nature of the construct of 
technological literacy that should determine 
the mode and conditions of its assessment. 
Assessment tasks should elicit from students 
the knowledge, behaviors, and attitudes that 
are believed to be characteristic of a techno¬ 


logically literate person. The multidimensional 
nature of TL makes the design of such tasks a 
difficult proposition. 

Dyrenfurth (1988) reported on the use of a 
paper and pencil test of TL that attempted to 
assess some domain knowledge and some of 
the attitudes thought to be indicative of TL. 
Clearly such measures are of limited useful¬ 
ness given the strong element of practical 
capability that TL involves. Snow (1993) sug¬ 
gested that multiple-choice items and student 
portfolios represent opposite ends of a con¬ 
tinuum of response structure, and Messick 
(1994) implied that a mix of assessment strat¬ 
egies that includes structured exercises and 
open-ended performance tasks might be use¬ 
ful for achieving breadth of coverage within a 
domain. The implications of these views are 
that multiple strategies, rather than a single 
mode, are more likely to tap the range of 
competencies included in a domain. For as¬ 
sessment to be considered authentic, how¬ 
ever, there are other criteria besides breadth of 
coverage that must be met. Linn et al. (1991) 
remind us of the demand for meaningfulness. 
All forms of structured items, particularly 
multiple-choice, and to a lesser extent semi- 
structured ones, run the risk of being trivial 
and lacking in meaningfulness to students 
since they portray items of knowledge as dis¬ 
connected from the configuration of which 
they are a part. This seems to run counter to the 
notion of authentic assessment. It is critical 
that such items be seen as useful only as part 
of a wider range of assessment procedures and 
applicable principally to the knowledge do¬ 
main of TL. 

Perhaps the most important educational 
function of assessment is its formative or diag¬ 
nostic one. Authentic assessment should be 
designed to allow identification of students' 
needs. This requires that tasks be sufficiently 
open to allow students to display their unique 
understandings and capabilities so thatteach- 
ers can fashion or modify learning experi¬ 
ences to meet revealed deficiencies. This is 
particularly important in the assessment of TL. 
If, as Dyrenfurth (1991) noted, the develop¬ 
ment of TL proceeds along a continuum from 
"non-discernible to exceptionally proficient" 
(p. 140), then students will be situated at 
varying points along that continuum. Assess¬ 
ment tasks for TL should allow students to 
function at their most advanced points along 
the continuum. 

Consider also that the level of TL students 
achieve results from both planned curricular 
experiences in school and their out-of-school 
experiences with technology. Prime (1992) 
found this to be true for a sample of secondary 
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school students in Trinidad, whose experi¬ 
ences in the home environment was the larg¬ 
est determinant of their attitudes toward tech¬ 
nology. Thus, assessment tasks should allow 
students to display understandings drawn from 
both in-school and out-of-school experiences. 

Another implication of the progressive na¬ 
ture of the development of TL is that assess¬ 
ment must be ongoing and continuous over 
time, so as to depict changes in levels of 
capability and understanding as they emerge. 
Uses of portfolios, documentation, andgraphic 
presentations seem to be approaches that meet 
the need for uniqueness of expression and 
continuity of assessment. 

The growing interest in performance as¬ 
sessment is particularly important in the as¬ 
sessment of TL since technology is both pro¬ 
cess and product. The identification of needs, 
the generation of designs to meet those needs, 
the weighing of alternatives in the light of 
specifications, and the making and evaluation 
of products are some of the processes of tech¬ 
nology. Performance assessment, more accu¬ 
rately called performance-and-product assess¬ 
ment (Fitzpatrick & Morrison, 1971), is suited 
to assess both the skill component and the 
affective component of TL. The focus of such 
assessment should be the processes and the 
final product generated by students. Observa¬ 
tional techniques, teacher-student interviews, 
and the documentation of students' decision 
making while engaged in technological activ¬ 
ity serve to make visible the technological 
thinking which would finally be embodied in 
the finished product. Assessment of the prod- 
uctby predetermined and jointly agreed-upon 
criteria is an approach that draws both teacher 
and student into assessment procedures. Cata¬ 
loging students' reasons for design decisions 
and student-generated criteria for evaluating 
products reveal the values held by students. 
These become important assessment data. 

The fact that TL is essentially about func¬ 
tional competencies in the real world may be 
the source of the greatest assessment chal¬ 
lenge, that is, to design assessment tasks that 
incorporate the salient elements of the real 
world in which TL is actually displayed. School 
assessment, even performance assessment, 
runs the risk of being too formalized and 
decontextualized to provide evidence about 
real-world functional competencies, in which 
case they may tell little of what a student is 
likely to be able to do in the real-life context. 
In the real world, technological activity is 
always comprehensive and purposeful, and 
implies the ability to see opportunities to im¬ 
prove some aspect of the world either by 
creating something new or by making an 


existing thing better. It involves moving from 
the initial idea through successive refinements 
to a solution. This aspect of competence is 
easy enough to assess through performance/ 
product assessment. Since values and inten¬ 
tions are inescapable inputs into these activi¬ 
ties, competence involvesthecontinuous reso¬ 
lution of conflicting values. But values and 
intentions are not observable in the finished 
product and they are often neglected in assess¬ 
ment. Assessment techniques need to be de¬ 
veloped to overcome this difficulty. The use of 
teacher-student interviews, peer interactions, 
and the development of student profiles dur¬ 
ing the course of the student's engagement 
with a technological task hold some promise 
in this direction. The "comprehensiveness" 
aspect of technological activity implies a de¬ 
gree of commitment and ownership of the task 
that is difficult to capture in school assess¬ 
ment, where thetas ks setare often butsnippets 
ofthefull range of activities in a technological 
context. Given the iterative natureof the phases 
of technological activity, it might well be that 
performance at any phase carried out in isola¬ 
tion would be different from what it would be 
if done in the context of the whole process. 

Often people other than the creators deter¬ 
mine the success of a technology. This is 
certainly true in a commercial setting where 
the consumer often determines success. Func¬ 
tional competence thus implies a sensitivity to 
the humanness of technology and, more spe¬ 
cifically, to consumer issues. If one assesses 
students' capabilities in evaluating technol¬ 
ogy out of the context or contact with real 
clients, a vital aspect of real-world functional 
capability may not be measured or realized. 

In a sense, a technological activity is never 
completed. The "final" product is really noth¬ 
ing more than the most recent prototype. A 
student's ability to visualize new possibilities 
for refinement of a product is an aspect of 
capability that is difficult to assess within the 
time constraints usually imposed on school 
assessmentprocedures. Consider also that real- 
world technological performance is rarely done 
solo. The ability to function as part of a team 
and the ability to communicate ideas in a 
variety of modes such as in discussion, graphic 
presentations, and 3-dimensional models are 
critical aspects of functional capability. In 
such instances, group assessment tasks would 
provide more accurate reflections of real tech¬ 
nological activity and would promote the so¬ 
cial interaction from which students derive 
emotional support. 

The series of appropriate TL assessment 
approaches presented here embody many of 
the principles of illuminative assessment. 
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Multiple strategies which employ concrete 
activities that are relevant to the lives of stu¬ 
dents and are grounded in the real world are 
advocated here. The balance in such assess¬ 
ment is clearly on the side of the processes 
rather than on the products, and the activities 
rather than the outcomes of students' techno¬ 
logical work. While product assessment is 
important, products alone fail to exhibit the 
complex ongoing interaction of idea and ac¬ 
tion that lie at the heart of technological capa¬ 
bility and infuses all stages of technological 
activity. Even performance-oriented assess¬ 
ment will fall short of its goal unless the design 
of such assessment strategies is informed by a 
careful analysis of the elements of real-world 
functional competency, some of which have 
been suggested. These approaches produce a 
high level of student engagement with tasks, 
and blur the lines between learning activities 
and evaluation. In such situations, assessment 
becomes an integral part of the instructional 
process and exerts its most positive influence 
on teaching and learning. 

The Place of Validity 

The essential validity question is, How ap¬ 
propriate are the data produced by assessment 
for the purpose for which the assessment is 
intended? Messick (1994) proposed a range of 
validity criteria that includes the "content, 
substantive, structural, external, and 
generalizability aspects of construct-validity" 
(p. 13), which he suggested are applicable to 
all forms of assessment. Additionally, 
Frederikson and Collins (1989) and Linn, Baker, 
and Dunbar (1991) proposed that specialized 
validity criteria are needed for performance 
assessment. These include content quality, 
content coverage, cognitive complexity, mean¬ 
ingfulness, cost and efficiency, transfer and 
generalizability, fairness and consequences 
(Linn et al., 1991), and scope, reliability, and 
transparency (Frederikson & Collins, 1989). 

I suggest that assessment of TL, of which 
capability is an element, might best be done 
through procedures that are open-ended 
enough to allow for individual expressions of 
competence, that may have individualized 
criteria of worth, and that allow for the recog¬ 


nition of knowledge and skills that are not 
necessarily part of the planned curricular ex¬ 
periences of students. The notion of common 
tasks and standards for assessment is an inte¬ 
gral part of traditional methods of assessment 
and is critical to traditional concepts of valid¬ 
ity. Furthermore, the idea of rewarding knowl¬ 
edge and ski I Is outside of the planned curricu¬ 
lum runs counter to the traditional notion of 
content-validity criteria. 

Another specialized criterion that might be 
applicable is context validity. The closer an 
assessment activity is to the real situation to 
which its results are to be generalized the 
more valid the assessment is likelytobe. Inthe 
assessment of TL, context validity may be 
achieved by performance tasks that place chil¬ 
dren in real situations or by school-based tasks 
that tap a large number of real-world compe¬ 
tencies. This aspect of assessment becomes 
more important when one bears in mind the 
fact that it is not the performance itself which 
is the real object of assessment, but the com¬ 
petence which it implies—a competence that 
is only meaningful in the real world. 

Assessment of TL is a challenging task made 
so by its multidimensional nature. Appropri¬ 
ate strategies should involve multiple ap¬ 
proaches yielding multiple types of data. As¬ 
sessment should be open-ended to allow for 
the unique expression of individual achieve¬ 
ment. It should be ongoing and continuous. 

The use of portfolios, student documenta¬ 
tion of activity, graphic displays, and product 
demonstrations are useful techniques. They 
are, however, limited in that they are passive 
manifestations of capability. The more active 
manifestations that are indicative of the com¬ 
plex interaction of values, intentions, knowl¬ 
edge, and skills are perhaps better tapped by 
means of teacher-student conversations, peer 
group discussions, and observational tech¬ 
niques conducted during student project work. 
Together these approaches seem likely to al¬ 
low students to make public their own unique 
capabilities. The consequence of such strate¬ 
gies, which according to Messick (1994) is itself 
a validity issue, will be the enhancement of the 
learners' engagement with technology, thus 
increasing their capability and competence. 
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